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Abstract. We investigate the relations between the structure of the image and events in 
the geometry of the undelying surface. We introduce some elementary differential geometry 
and use it to define a coordinate system on the object based on the lines of curvature. 
Using this coordinate system we can prove results connecting the extrema, ridges and 
zero-crossings in the image to geometrical features of the object. We show that extrema of 
the image typically correspond to points on the surface with zero Gaussian curvature and 
that parabolic lines often give rise to ridges, or valleys, in the image intensity. We show that 
directional zero-crossings of the image along the lines of curvature generally correspond to 
extrema of curvature along such lines. 

© Massachusetts Institute of Technology, 1984 



This report describes research done at the Artificial Intelligence Laboratory of the Massa- 
chusetts Institute of Technology. Support for the laboratory's artificial intelligence research 
is provided in part by the Advanced Research Projects Agency of the Department of Defense 
under Office of Naval Research contract N00014-80-C-0505. 



-j<^*\ 



/""S 



o 



^*\ 



Zero Crossings on Lines of Curvature 



1. Introduction 

The aim of this paper is to investigate how much information can be found from an image 
when the reflectance function is unknown. More precisely we will assume that the image 
E(x) is determined from the reflectance function R{p,q) and the albedo a(x, y) by the image 
irradiance equation (Horn 1977) 

E{x l y) = a{x,y)R{p,q). (1.1) 

For a surface z = f(x, y) p = df/dx and q — df/dy. This equation is a good description for 
the reflectance of many surfaces (Horn 1977). For many surfaces, including most of those 
considered in shape from shading, the albedo is constant or varies slowly. We will assume 
it is constant in this paper although many results will not change much if we allow it to 
vary slowly. Equation (1.1) can be modified to hold for specular surfaces but we will not 
consider such surfaces in this paper. 

For the 1 -dimensional case we have 

E{x) = R{p). (1.2) 

Suppose we know nothing about R(p) except that it is continuous. It is still possible to 
obtain information about the surface by observing that R is a function of p only. If there is 
a discontinuity in the surface gradient p (i.e. a 2 st order discontinuity in the surface) there 
will be a I st order discontinuity in the image. In general an N th order discontinuity in the 
surface corresponds to a (N - 1)"' order discontinuity in the image. 

We extend this type of arguement to continuous surfaces and prove results relating the 
distributions of extrema and zero-crossings in the image to features of the underlying surface 
geometry. To do this we will need to introduce differential geometry concepts such as lines 
of curvature. We assume that the surfaces of the objects are sufficiently smooth for the 
lines of curvature to be defined everywhere. We also assume the image is not smoothed by 
a filter. 

In recent years workers in vision have shown considerable interest in the principal lines 
of curvature of surfaces. For example Curvature Patches have been proposed as a 
representation for visible surfaces (Brady 1983, Brady and Yuille 1984) and there exist 
various schemes for dividing objects into parts based on extrema and zeros of curvature 
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(Brady 1983, Hoffman 1983, Hollerbach 1975). There is also some evidence from line 
drawings that curves in an image are interpreted as lines of curvature (Stevens 1979) of a 
perceived surface. 

On the other hand it has been suggested that the principal lines of curvature of a 
surface can only be computed indirectly (for example from edge detection, followed by stereo 
and surface reconstruction) and with great difficulty. The complexity of the calculations also 
implies bad numerical behaviour and excessive sensitivity to noise. Recent experimental 
results (Terzopoulos 1984, Brady and Yuille 1984), admittedly using laser data suggest, 
however, that these lines can be determined reliably. 

This paper is divided into two parts. First we set up a coordinate system based on lines 
of curvature and relate the image directly to the underlying geometry of the surface using 
the image irradiance equation (Horn 1977). Then we prove some results concerning zero 
crossings and the principal lines of curvature. These imply that in certain circumstances we 
can obtain information about the principal lines of curvature, at least approximately, directly 
from the image. 

In the second part of the paper we consider the relation between the extrema of the 
image and the parabolic lines of the surface (the lines on which the Gaussian curvature 
vanishes). Using the line of curvature coordinate system we can generalize results obtained 
by Koenderink and van Doom (1980). We show that almost all extrema of the image lie 
on parabolic lines and that the isophotes (the lines of constant image intensity) point along 
the lines of curvature when they cut the parabolic lines, independent of the details of the 
reflectance function. We then argue that parabolic lines typically give rise to ridges, or 
valleys, in the image intensity. For completeness, in Appendix (1), we include an alternative 
proof of the first of Koenderink and van Doom's results which does not rely on the line of 
curvature coordinate system. We also include an example to show that some extrema of 
the image intensity do not lie on parabolic lines. 

We illustrate these results with the following example. Consider the bottle shown 
in figure (1). Its curvature will be zero at the head and the base. The neck is initially 
convex, passes through an inflection point and then becomes concave. Thus the curvature 
is positive, zero and then negative. The results state that the line with zero curvature will 
correspond to a ridge, or valley, in the image intensity and that there will be two rings of 
zero crossings. Furthermore, provided there is not too much foreshortening, these rings are 
likely to be near the extrema of curvature. 

We start by considering various directional zero crossing operators. We show that 

/•■n, directional zero crossings do not necessarily correspond to physical, or observable, zero 

crossings. By physical zero crossings we mean those that correspond to sharp changes in 
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the image irradiance. We suggest that directional zero crossings are physical only if their 
/**\ direction is along the line of greatest change of the image irradiance and prove a result 

supporting this choice of operator. This directional operator has been discussed by various 
authors (Havens and Strikwerda, 1983). More recently Torre and Poggio (Torre and Poggio 
1983) have considered it from an alternative point of view. Canny (Canny 1983) has argued 
for this operator from the standpoint of signal processing. 

In section 3 we introduce some mathematical notation and recall some results from 
Differential Geometry about principal lines of curvature. 

Next, in section 4, we introduce the image irradiance equation (Horn 1977). A 
probabilistic argument shows that the directions of greatest change of the image irradiance 
are most likely to be along the tines of principal curvature. This suggests that many of the 
physical zero crossings are directional zero crossings along the principal lines of curvature 
of the surface that is imaged. 

In section 5 we prove some results about the distribution of zero crossings along lines 
of. curvature. Our starting point is the work of Grimson on surface consistency (Grimson 
1981). With relatively weak assumptions about the reflectance function (Horn 1977) he was 
able to produce neccessary and sufficient conditions in one dimension for the occurence of 
directional zero crossings in the image irradiance in terms of the surface geometry. He then 
used some probabilistic assumptions about the reflectance surface to extend this result to 
two dimensions and prove his Surface Consistency Theorem. This theorem was the basis 
for his theory of surface interpolation. 

We present an alternative development of his analysis, and derive some significant 
extensions. For example, we are able to derive, without probabilistic assumptions, necessary 
and sufficient conditions for the occurence of directional zero crossings along principal lines 
of curvature. We call this result the Line of Curvature Theorem. It states that there will 
be a directional zero crossing along a line of curvature between points where the principal 
curvature of that line vanishes. Moreover, depending on the reflectance function, this zero 
crossing is likely to be near the extremum of the principal curvature. 

In conjunction with the results of section 4, the Line of Curvature Theorem suggests 
that many, if not most, of the physical zero crossings can be associated with points on 
the lines of principal curvature which are near the extrema of the principal curvatures (see 
Figure 1). This supports the view that it may be possible to obtain an approximation to the 
lines of principal curvature directly from the image. It suggests that part boundaries at the 
extrema of curvature can be found at a low level of visual processing. Finally, it provides 
additional support for representations of shape based on principal lines of curvature (Brady 
1983) and shows that these coordinates are useful for analyzing the image intensities of 
surfaces. 
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Figure 1. 1. Illustrations of the Line of Curvature Theorem. In (a) we show a bottle. The theorem 
implies that there is a zero crossing along each of the parallels. Since the bottle is a surface of 
revolution, it has two rings of zero crossings around the collar. Similarly, (b) is a singly ruled surface 
for which the zero crossings lie along rulings. 

The principle result of the first part cf this paper is the Line of Curvature Theorem. 
This derives results about the distribution of directional zero-crossings along the lines of 
curvature. Assuming that the zero-crossing operator is the directional derivative along the 
intensity gradient we then argue that these zero-crossings are physical. If another zero- 
crossing operator is chosen, the Laplacian for example, these arguments are not directly 
applicable. Instead we argue that the difference in the zero-crossings of different operators 
(provided they are reasonably similar) lies in the positions of the zero-crossings and not 
in their existence. If results like the condition of linear variation apply (Marr and Hildreth, 
1980) the change in the positions of the zero-crossings should be small. 

In section 6 we begin the second part of the paper. Using the mathematical machinery 
already set up it is straightforward to extend the results obtained by Koenderink and 
van Doom (1980).The most interesting extension is the claim that parabolic lines often 
correspond to ridges in the image intensity. In Appendix 1 we provide an alternative proof 
of some of Koenderink and van Doom's results which makes no use of the curvature 
coordinates. We also provide an example to show that extrema of the image do not always 
lie on parabolic lines. 

2. Physical Zero Crossings 
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We will now show that not all directional zero crossings can correspond to physical 
^^ zero crossings. As before we use "physical" to mean those zero crossings that correspond 

to sharp intensity changes in the image. 

Most zero crossing detection operators involve filtering the image with a suitable 
function, usually a Gaussian. In this section, and for the rest of the paper, we will not 
consider the effect of the filter in any detail. The purpose of the filter is to smooth the 
image, thereby isolating irradiance changes at a particular scale. Hence its only effect will 
be to determine the scale at which the image is analysed and to rule out fluctuations in the 
image below this scale. 

We now show that there are directional zero crossings everywhere in the image. More 
precisely any circle G p in the image plane with arbitrary centre and radius p will contain a 
directional zero crossing. Note that we are considering continuous images, which have not 
been digitized. 

Let E be the image irradiance. Pick two points in C p with different values of- E. At 
each point there must be a direction in which E does not vary. Join the points by a curve 
in C p whose tangent vector at each endpoint is in the direction in which E does not vary. 
There is always an uncountably infinite number of such curves. Now consider the derivative 
f\ of E along the curve. It is zero at the endpoints and must be non-zero somewhere along 

the curve since the endpoints have different values of E. Hence it must have an extremum 
somewhere on the curve and this will be a directional zero crossing in C p . Allowing p to 
tend to zero proves that the set of directional zero crossings is dense in the image plane. 

From this argument it is clear that not all directional zero crossings are physically 
significant. If a curve has unit tangent vector I'' then a directional zero crossing is a 
solution of 

(/"V M )(rv t/ )i? = I (2.1) 

where V ;i denotes the derivative operator. For example in cartesian coordinates V,, = 
(zfe>^)- We use tne summation convention on the suffices \i,v and so /''V,, denotes the 
scalar product of the vector l 1 ' with the gradient operator V M (Hawking and Ellis, 1973). We 
can write (2.1) as, an alternative version is given in Appendix 2, 
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i"/"(V M V^)/!? + J M (V„nV„tf - 0. (2.2) 

If /'' is a constant vector, as in Grimson's work, the second term vanishes. However 
in general it is non zero though it seems plausible to require that the second term in (2.2) 
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Figure 2. 2. Directional zero crossings. In (a) the unit vector points across an intensity boundary 
and the zero crossing is physical. In (b) the unit vector points in an arbitrary direction but a curve 
can be choosen so that it corresponds to a directional zero crossing. 

vanishes, or is small. This will ensure that the zero crossing does not depend too much on 
the derivative of the tangent vector l* (in practice this problem rarely arises since we tend to 
convolve an image in a set of constant directions). To see the importance of this consider 
figure (2). The physical notion of a directional zero crossing is illustrated in (2. a). The unit 
vector points from a region of light to a region of dark and is approximately perpendicular 
to the boundary of these regions. In figure (2.b) however the unit vector f at x points in 
an arbitrary direction in a region of almost uniform irradiance. Now there are an infinite 
number of curves though x with tangent V. The result above tells us that it is almost always 
possible to choose a curve to get a zero crossing at x (given V at x one finds the curve by 
solving (2.2) for V^r). This zero crossing arises because of the derivative of the tangent 
vector and is clearly not physical. 

We write this constraint as 



P( V)V„£ - 0. 



(2-3) 



The most intuitive zero crossing operator is the directional derivative along the direction 
of the irradiance gradient (Canny 1983, Havens and Strikwerda 1983, Torre and Poggio 
1983). This corresponds to a directional derivative 
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where \V„E\ is the modulus of the vector V V E. In other words we choose / M to be 
^~\ jv7k|( v ^)- Hence 

d 4 - l V ^l (2-5) 
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eft 2 5a;j dxjdxk dx],\dxi d 
A feature of this operator is that it automatically satisfies our requirement since 



WrvbiMh* - ^Misbf^rvin''*} - °- (2 - 7) 

This result can be used to argue both for (V„E)V„ as the directional operator and for 
our requirement. It means that the zero crossings of this operator will not depend on the 
derivative of the tangent vector. 

Recent work by Canny (Canny 1983) supports this choice of operator. He specifies 
three performance criteria and looks for the optimal edge detection operator in the presence 
of background noise. He has also successfully implemented this operator on a large range 
of images and compared its performance to that of other operators. 

3. Lines of Curvature 

In this section we briefly recall some of the basic concepts of differential geometry for 
use in later sections. Consider a surface given by 



* = /(*,»)■ (3.1) 



We can write this as 



r = (i, »,/(«,»)). (3.2) 

We will use suffix notation, (x,y) = [x u x 2 ), and write a three dimensional vector I as 

\=*(h,l.) (3.3) 

splitting it into its components in the [x, y) plane and its component l a in the z direction. 
/""■% We use the summation convention so if a suffix appears twice it will automatically be 

summed. For example we use l a m a to denote J2 a l a m a . 
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The unit normal to the surface is 



\ dxj dxj J \ dxi ' J 



(3.4) 



For a surface parametrized by u and v we define the first and second fundamental 
matrices of the surface by 



G = 



SI . SI SI . SI 

du du dv du 

sr . ar or . ar 

,Uv du dv Ifv 



(3.5) 



and 
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a 3 r_ 

dudv 
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a 3 r 

dudv 

all 



(3.6) 



Consider a curve u = u(t) on the surface r — r(u,v). Let D and G be the second and 
first fundamental matrices of the surface. Then the lines of curvature (Faux and Pratt, 1979) 
are the solutions of the equation 



(D-/cG) 



di} 

It 



0. 



(3.7) 



At any point there are two directions of curvature with curvatures ki and k 2 . They are 
automatically mutually orthogonal, since they are the eigenvectors of the same matrices, 
except at umbilic points where *i = « 2 . However at these points it is still possible to choose 
them to be orthogonal. The Gaussian curvature G is the product of the two curvatures. 



G = K1K2. 



(3.8) 



Now choose the parametrization of the surface so that u and v are parameters along the 
lines of curvature. This corresponds to choosing a "curvature webbing" (Brady 1983) and 
is illustrated in figure(3). The directions of curvature are dii/du and dxi/dv with curvatures 
k u and k v . 

It is easy to see from (3.5) and (3.6) that 



dr dx „ 
du ov 



(3.9) 



and 
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Figure 3. 3. A Line of Curvature webbing. 



d 2 r 
dudv 



0. 



(3.10) 
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Equations (3.9) and (3.10) demonstrate the chief mathematical advantages of a curva- 
ture patch representation (Brady 1983). They show that it diagonalizes the first and second 
fundamental matrices of the surface, see (3.5) and (3.6), and corresponds to the "flattest" 
possible local coordinate system. 



For a curvature webbing we can write down the principal curvatures directly from (3.7), 
namely 



<9 2 r ^ Dr_ dx_ 

du 2 du du 



(3.11) 



and 



d 2 r dt dr „ 
n- -^-o-Kv—- — = 0. 
av i av av 



(3.12) 



We write r — (x,-,/) and using 
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dj_ df du 
du dxi du 



we rewrite (3. 9), (3. 10), (3.11) and (3.12) as 



(3.13) 
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dxidxj du dv 



(3.15) 



dx 



> 2 / dxjdxj + / + df_df\'(dldf_ + 6 \dx i dx l = Q ,3 J6) 

idxj du du u \ dxkdxk) \dx{ dxj i3 ) du du 



dx 



> 2 / dxidxj { r | d/ d/\i(df df | ^ \d Xi d Xj ^ Q 

,-3xj dv dv "v. dxkdxit) \dx( , dxj x] ) dv dv ' 



Note that since dxi/du and dx,/du are independent for a curvature webbing, we can 
use (0.14) and (3.15) to write (3.16) and (3.17) as 



_d 

dx 



>'/ axj | r } df df\*fdf df | v \a*< _ (318) 

idxj du u \ dxkdxk) \dx{ dxj U J <9u 



d 2 f d^ | / | a/ g/_\' |£/ a/ | f ..|g»< =0 (319) 

dxidxj dv v \ dx^dxkj \dxi dxj l3 ) dv 



These are the main results we need about lines of curvature. We finish this section 
with a few remarks about derivatives on surfaces and Christoffel symbols. 

Suppose we have a vector field on a surface defined relative to some coordinate 
system. Unless the surface is flat, the directions of the axes of the coordinate system will 
vary over the surface. This also happens if the surface is flat and we choose a coordinate 
system other than the usual cartesian one, for example Polar Coordinates. In either case 
we must take the variation of the axes into account when we differentiate. To do this we 
use Christoffel symbols (Millman and Parker 1977, Hawking and Ellis 1973) denoted by T£ p . 
The derivative of a vector / M in a coordinate system {xi,x 2 ) is then 

4. The Directions of the Irradiance Gradient 

We now argue that the most likely directions of the irradiance gradient are along the 
lines of curvature. 

/f~\ Consider the image irradiance equation (Horn 1977). We assume orthographic projec- 

tion onto the {x, y) plane and write the equation as 
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^ E{ Xi ) = J2(/,-) (4.1) 

where /,• = df/dxi, i — 1,2. The albedo is assumed constant although the results still 
hold if it varies slowly. 

A vector /,- in the surface has unit length if it satisfies 

while a unit vector m, in the image plane obeys 

rmmi = 1. (4.3) 

If we consider vectors along the directions of curvature that have unit length in the 
surface we find from (3.15) and (3.16) 
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d 2 f dxt dxi , , 

" - J — -kJ 1 + 



OXiOXj OU OU \ OXk OXkJ 

Jx{dxj dv dv \ dxk dxk J 



We also have 



d~f Oxi dxj 
dx{dxj du dv 

and the orthogonality condition 



= (4.6) 



( °* df U, ) dXi dXj = (4.7) 

\ (9s,- Oxj n J du dv 

Thus Oxi/du and dx { /dv have unit length in the surface and are the directions in the 
surface for which the matrix with coefficients -g§£~ ' s extremized (equivalently they are the 
directions of curvature). Note we can think of these directions as those which extremize 



11 



YUILLE 



where A can be thought of as a Lagrange multiplier used to impose orthogonality. The 
standard equations for lines of curvature can be obtained from (4.8) by using the Euler 
Lagrange equations. 

We can write the derivatives of E in direction i, as 

>>¥■ - winr 1 '- <"> 

axi ofj oxidxj 

The irradiance gradient will be in the direction U which maximizes this expression. Now 
suppose that dR/dfj is in a random direction, or that we do not know what the reflectance 
function is. Then the best choice for /,• is that which extremizes 

where the vectors /,■ have unit length in the image plane while the curvature directions 
dxijdu and dxijdv which extremize ^J- have unit length in the surface 

l^LK. + St X^i^i = i (4.ii) 

\0xidxj U J du du K ' 

( axi oxj J ov av 

However if we average over the whole surface ^|~ and §f ( ^j- should be equal. 
Hence on average the lengths of dxt/du and dxi/dv in the image plane will be equal and 
they will be the best choices for /,. Thus the most likely directions for the gradient of the 
image irradiance is along the lines of curvature. Note that the most likely directions of V V E 
are not the same as its average direction. 

5. The Line of Curvature Theorem 

in one dimension the image irradiance equation can be written 



E(x) = R(p) (5.1) 

where p = df/dx. Grimson (1 981 ) obtains his theorem in one dimension by differentiating 
to obtain 

jf»~»> dE. dR dp ,_ ... 

w \ — =■ — — — (5.2) 

dx dp dx 
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and observing that zero-crossings lie between points where dE/dx — which typically 
f\ occur at places where dp/dx — 0. In two dimensions this argument fails since the right hand 

side of (5.2) will now contain several terms. We shall show, however, that it is possible to 
extend this proof to two dimensions by a special choice of coordinates. 

Observe that in the flat two dimensional image space (x,) and (/,-) are both vectors 
and equation (4.1) is true whatever coordinate system we choose to represent them in. 
The coordinate system based on the lines of curvature is a natural choice. We define 
coordinates (m,u 2 ) = (u,v) and calculate the Christoffel symbols (Millman and Parker 1977, 
Hawking and Ellis 1973). 

It should be emphasized that the space we are considering is the flat two dimensional 
(x, y) plane rather than the surface described by (2.1). This is because zero crossings are 
found in the image plane. 

Denote the differential operator by V M . In this coordinate system we find (/,•) to be 
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The second derivatives of / are 

V,V M / = ^(V M /)-r^(V,/). (5.5) 

Setting u — ^i — \ and using (5.3) we obtain 



dV _ d_(Of\ 2 xi df 
du 2 du ' 



l{du)~jS"dx~ i - (56) 

Using (3.13) we can simplify this as 



Similarly we find 



d 2 J__ d 2 f dxjdxj 
du 2 dxidxj du du 



d 2 f _ 2 f dxjdxj 
dv 2 dxidxj dv dv ' 



(5.7) 



(5.8) 



and 
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^■% dudv dxidxj du dv 
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Now using (3.15) we find 

rf 2 / 



dudv 



0. (5.10) 



This result is very important since it enables us to generalize Grimson's one dimensional 
theorem (Grimson 1981). We write the derivative of E in the u direction as 



dE^dR^dU OR^df^ 
du df u du df v du 



Using (5.10) this becomes 



dE^tmdU 12) 

du df u du' 

Then Rolle's Theorem tells us there is a zero of d 2 Ejdu 2 , or equivalent!;/ a directional 
zero crossing, in between points where dEjdu — 0. 

We would like a result that is independent of the reflectance map R and, following 
Grimson, we assume that dR/df u varies slowly compared with df u /du. This assumption is 
valid for many forms of the reflectance map (Grimson 1981). Then zeros of dEjdu are likely 
to be zeros of df u /du. Using (5.7) and (4.4) we see that these occur at 



k u =■■ 0. (5.13) 

We have proved the Line of Curvature Theorem which states that there will be a 
directional zero crossing along a line of curvature between points where the principal 
curvature of that line vanishes. These are the only zero crossings along such lines. 
Furthermore since the zero crossing occurs at an extremum of dEjdu it is likely to be near 
an extremum of dj u jdu. So it is likely to be near an extremum of the curvature *„. The 
more uniform the Reflectance function is the better these assumptions will be. Calculations 
suggest that these assumptions are most accurate when the gradient of the surface at the 
extremum of curvature is small. This will be investigated experimentally. Examples of this 
Theorem are shown in figure (1). 

It remains to show that these zero crossings are physical. To do this we can use the 
results of section (4) to argue that zero crossings along the lines of curvature are likely to 
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be along the direction of the irradiance gradient. Alternatively we can see whether the lines 
of curvature satisfy the requirement (2.3). For lines of curvature we calculate 



Near extrema of the curvature we expect this to be small since dE/dx { is likely to be 
in the direction dx { /du and the unit length of the tangent vector implies 



d 2 Xi dx{ 

3^ = - < 5 ' 15 ) 



6. Extrema of the Irradiance 



In a recent paper Koenderink and van Doom (1980) proved that the extrema of 
illumination of an object lie on the parabolic lines (the lines on which the Gaussian curvature 
vanishes) and moreover that the isophotes (the lines of constant illumination) cut the 
parabolic lines at angles which are independent of the position of the source. Using the 
notation developed in the previous sections we are able to prove stronger results. 

We first show that the extrema of illumination almost always lie on the parabolic lines 
f^: (we demonstrate an exception in Appendix 1) and show how their exact position depends 

on the reflectance function. Secondly we show that the isophotes lie along the lines of 
curvature at parabolic lines and hence that angles at which they cut the parabolic lines is 
a property only of the geometry of the surface and the viewing position and is independent 
of of the reflectance function and the position of the light source. Finally we argue that 
these parabolic lines typically correspond to ridges, or valleys, in the image intensity and so 
it might be possible to compute them directly from the image. 

The parabolic lines of a surface are those for which the Gaussian curvature vanishes. 
Hence they divide the surface into regions of positive and negative Gaussian curvature. 
Their apparent importance for shape description led the mathematician F. Klein to use them 
in his attempt to show that the artistic beauty of a face was based on mathematical relations 
(Hilbert and Cohn-Vossen, 1952). 

From (3.8) we see that a necessary and sufficent condition for a line to be parabolic 
is that at least one of the principal curvatures must vanish on it, for example k u = 0. 

The extrema of irradiance occur when 



V u £=.-V„/i,' = 0. (6.1) 
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From equations (5.12), (5.7) and (3.16), removing terms which are positive definite, we 
see that (6.1) is equivalent to 

dR dR 

37«u = jrK» = 0. 6.2) 

Hence umbilic points with zero Gaussian curvature (« u = *c„ = 0), called locally flat 
points, will give rise to extrema of the irradiance. These points lie on parabolic lines. The 
most typical extrema are those where k u — and §jt = (and the same with u and v 
reversed). These lie on parabolic lines but their exact position will depend on |^ and hence 
on the reflectance function and the positions of the viewer and the source. A final, but 
unlikely, possibility is when §^ = f^- = 0. This will rarely, if ever, occur on a parabolic 
line. 

Note that extrema of the irradiance do not necessarily correspond to extrema of the 
surface. These occur at 



/,• = 0, (6.3) 

and from (4.1) we see that E will have the same value at all these points 

E = 11(0). (6.4) 

Whether this corresponds to an extrema of the irradiance or not depends on the explicit 
reflectance map. For a Lambertian surface 



m = a + \*m?,m* (6 - 5) 

(1 +g;<7 J >(l + fkfkP 
where the vector q { corresponds to the direction of the source. 



11(0) = (6.6) 

(1 + q iqi )l 

and 



V + Wi)* 
Near an extremum of / (£/,)</,- can take any sign and so 11(0) is not an extremum. 



16 



YUILLE 



f^ Now suppose the reflectance map depends only on the angle between the surface 

normal and the observer, perhaps corresponding to a uniform distribution of sources. Then, 
for some number n, 

*(/.) = (! + /,/;)" (6-8) 

which will clearly be extremized at /,- = 0. 

We now consider the second result of Koenderink and van Doom (1980). The isophotes 
are the lines of constant illumination and are similar to the contours on a relief map. At any 
point their direction is given by the vector / such that 



l'VE = (6.9) 

Suppose we are on a parabolic line given by k u = 0, Using (6.2) and (6.1) it follows 
that the gradient of the illumination in the direction of the line of curvature parameterized 
by u is zero, V U E *= 0, on the parabolic line. Hence on parabolic lines the isophotes point 
along the lines of curvature. As a corollary of this result we deduce that the angles at 
which the parabolic lines cut the isophotes is independent of the reflectance function and 
the position of the source. 

We now argue that the parabolic lines generally correspond to ridges or valleys of the 
image illumination. Again we consider a parabolic line with k u = 0. As shown above this 
corresponds to V U E = 0. Now consider the values of V U E on either side of this line. Since 
the line is parabolic with k u — it follows that k u will have different signs on either side of 
the line, without loss of generality k u will be positive on the left side of the line and negative 
on the right hand side. From (5.12), (5.7) and (3.16) we see that the sign of V U E on the left 
and right hand sides of the line will be respectively minus or plus the sign of $#•. In the 
generic case §f- will be non-zero on the parabolic line and so will have the same sign on 
either side of the line, for example it could be positive on both sides. Moreover it is likely 
that this sign will stay the same as we travel along the curve. Provided these assumptions 
are valid V V E will be negative on the left side of the line, zero on the line and positive on 
the right side. Thus, provided the lines of curvature are not too badly behaved, the second 
derivative of the intensity in the direction of the line of curvature taken at the intersection 
with the parabolic line will have the same sign along the curve and so the the curve will 
correspond to a ridge or valley in the image intensity. 

7. Conclusion 

We have argued that many significant zero crossings lie on the principal lines of 
curvature near the extrema of curvature. This suggests that it may be a lot easier to 



^~N ( 
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compute some lines of curvature than is commonly believed and that curvature patches 
f^, are a natural representation for surfaces. Furthermore if the zero crossings pick out the 

extrema of the principal curvatures it might be possible to make a rough reconstruction of 
the surface from them. 

These results also suggest that part boundaries defined near extrema of the curvature 
occur near lines of zero crossings. Thus it may be possible to read the part boundaries 
proposed by Hoffman (1983) and Hoffman and Richards (1984) directly from the shading of 
an image. 

The extensions of Koenderink and van Doom's results (1980) rederived in section 6 
is also interesting as it suggests that the parabolic lines of the surface can be obtained 
directly from the image. It is suprising that extrema almost always lie on parabolic lines, 
that these lines are often ridges, or valleys, in the image intensity and that the isophotes 
point along the directions of curvature when they cut the parabolic lines. 

In conclusion it seems that information about the background geometry, and in 
particular about the lines of curvature and the parabolic lines, can be obtained from the 
image without too many assumptions about the reflectance function. A coordinate system 
based on lines of curvature seems a good way to represent a surface. 
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Appendix 1. 

We first give a short proof showing that the extrema of the image irradiance usually 
occur on parabolic lines. Then we consider a sphere with a lambertian reflectance function 
to demonstrate an exception. 

We write the image irradiance equation (4.1) in terms of cartesian coordinates 



We differentiate twice by parts, using p x = §| and p y = §£, to obtain 

Ox 0p x dx 2 dp y dxdy 

and 



™ == ™: d lL + ?JL l!L (Al .3) 

dy dp y dy 2 dp z dxDy' 
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In cartesian coordinates the second fundamental form can be written 



D 



■alt ~ a 'f 

" OzOy 9 dy' . 



{AAA) 



where , = (i + {^f + (§£) 2 )*. 

An extremum of the image irradiance occurs when 



dE_dE_ n 



dx 8ij 
From equations (A. 1,2), (A.1.3) and (A. 1.4) it follows that either 



dR OR n 

dp y dp x v ' 



or 



detD = 0. (A1.7) 

f^^ If (A.1 .7) holds then, by definition, the Gaussian curvature vanishes and so the extremum 

lies on a parabolic line. This will be the typical situation. However it is also possible that 
(A.1 .6) holds and the extremum lies off the parabolic lines. As an example consider a sphere 
with a Lambertian reflectance function. The sphere has constant Gaussian curvature and 
hence no parabolic lines. It must, however, have an extremum of illumination which must 
therefore correspond to (A.1. 7). It is straightforward to check this by an explicit calculation. 

Appendix 2. 

We illustrate the mathematics of section (2) with the following example. 

Consider a coordinate frame (X 1} Yi) rotated by an angle relative to a frame (X,Y). 
The unit vector in the Xi direction is / where 

/ = (cosO, ainO) (A2.1) 

We denote the gradient derivative operator by V where 

^ v = (A,A). („. 2 . 2) 

Now the transformations between frames is given by 
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Xx = XcosO + YsinO, (A.2.3) 



Y Y = 7cos0 - Xs*n0, (A.2.4) 



and 



X = X L coaO - YisinO, (A2.5) 



Y = YicoaO + X x ainO, (A2.6) 



We have 



= Z-V = «w0— + jW-, (yl.2.7) 



dXt dX dY 

Now a directional zero-crossing of E in the X x is a solution of 

0X2=0, (A2.8) 

or equivalently 

(I • V)(/ • V)E = 0. (A2.9) 

We use (A. 2.7) to expand this as 



m = Mgj: + „„*_)(«„*_ + „„*_). (.1.2.10) 

We allow to be a function of X and V, = 0(X,Y), so the rotation is no longer rigid. 
Now l(X,Y) will be a vector field whose direction depends on its position in space. We can 
use (A.2.10) to write —^ as the sum of two terms. 

2 E _ (cos 2 0E X x + 2ain0cos0E X Y + sin 2 0E Y y) - E x {sin0cos00 x 
dX\~ +sin 2 00 Y ) + E Y {cos 2 00x + sin0cos00 Y )' 

The second term on the right hand side of (A. 2.11) occurs because / is a function of 
(X, Y). In the notation of (2.1) the first term corresponds to /T(V m V„)/j' and the second 
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to /"(V^/")V^i?.ln section (2) we argue that unless this term is small a solution of (A. 2.8) 
f"*% will be an artifact of the vector field 1{X,Y) rather than a physical zero-crossing. We show 

in equation (2.7) that the second term always vanishes when / is choosen to be along the 
direction of the gradient of the image intensity. 

References 

1. J.M.Brady, "Criteria for representations of Shape," Human and Machine Vision, 
Beck, J., and Rosenfeld, A. (eds), Academic Press: New York, 1983. 

2. J.M.Brady, "Parts description and acquisition using vision," Robot Vision, ed. 
Rosenfeld, SPIE volume 336, 1982b, 20 • 28. 

3. J.M.Brady and A.L.Yuille, "Representing three dimensional shape," Romansy 
Conf, Udine, Italy, 1984, . 

4. J. F. Canny, "Finding edges and lines in images," MIT A. I. Lab Technical Report No. 
720 (1983), . 

5. I. D. Faux and M.J.Pratt, Computational Geometry for Design and Manufacture , 
Ellis Horwood Ltd, Chichester, England, 1979. 



/•"S 



6. W.E.L.Grimson, From Images to Surfaces , M.l.T.Press, Cambridge, Mass, 1981. 

7. S.W. Hawking and G.Ellis, The Large Scale Structure of Spacetime , Cambridge 
University Press, Cambridge, England, 1973. 

8. W.S. Havens and J.C.Strikwerda, "An Improved Operator for Edge Detection," 
To appear (1983), . 

9. D.Hilbert and S.Cohn-Vosseh, Geometry and the Imagination , Chelsea Publishing 
Company, New York, 1952. 

10. D.Hoffman, Representing shapes for visual recognition, MIT, PhD Thesis, 1983. 

11. D.Hoffman and W.Richards, Parts of Recognition, MIT, A.I. Memo 732, 1984. 

12. J.M.Hollerbach, Hierarchical shape description of objects by selection and 
modification of prototypes, MIT, also AI-TR-346, 1975. 

13. B. K.P.Horn, "Understanding image intensities," Artificial Intelligence 8 (1977), 
201-231. 

14. J.J.Koenderink and A. van Doom, "Photometric invariants related to solid 
shape," Optica Acta 27 (1980), 981-996. 

^•- 15. D.Marr and E.C.Hildreth, "Theory of Edge Detection," Proc.R.Soc.Lond B.207 

(1980), 187-217. 



21 



YUILLE 



16. R.S.Millman and G.D.Parker, Elements of Differential Geometry , Prentice-Hall, 
Englewood Cliffs, New Jersey, 1977. 

17. V.Torre and T.Poggio, "On Edge Detection," M.I.T. A.I. Memo 768 (1984), . 

18. K.A.Stevens, Surface perception from local analysis of texture (also, A.I.-TR-512), 
M.I.T, 1979. 



r^ 



fs 



22 



